INN Hotels Project¶
Context¶
A significant number of hotel bookings are called-off due to cancellations or no-shows. The typical reasons for cancellations include change of plans, scheduling conflicts, etc. This is often made easier by the option to do so free of charge or preferably at a low cost which is beneficial to hotel guests but it is a less desirable and possibly revenue-diminishing factor for hotels to deal with. Such losses are particularly high on last-minute cancellations.
The new technologies involving online booking channels have dramatically changed customers’ booking possibilities and behavior. This adds a further dimension to the challenge of how hotels handle cancellations, which are no longer limited to traditional booking and guest characteristics.
The cancellation of bookings impact a hotel on various fronts:
- Loss of resources (revenue) when the hotel cannot resell the room.
- Additional costs of distribution channels by increasing commissions or paying for publicity to help sell these rooms.
- Lowering prices last minute, so the hotel can resell a room, resulting in reducing the profit margin.
- Human resources to make arrangements for the guests.
Objective¶
The increasing number of cancellations calls for a Machine Learning based solution that can help in predicting which booking is likely to be canceled. INN Hotels Group has a chain of hotels in Portugal, they are facing problems with the high number of booking cancellations and have reached out to your firm for data-driven solutions. You as a data scientist have to analyze the data provided to find which factors have a high influence on booking cancellations, build a predictive model that can predict which booking is going to be canceled in advance, and help in formulating profitable policies for cancellations and refunds.
Data Description¶
The data contains the different attributes of customers' booking details. The detailed data dictionary is given below.
Data Dictionary
- Booking_ID: unique identifier of each booking
- no_of_adults: Number of adults
- no_of_children: Number of Children
- no_of_weekend_nights: Number of weekend nights (Saturday or Sunday) the guest stayed or booked to stay at the hotel
- no_of_week_nights: Number of week nights (Monday to Friday) the guest stayed or booked to stay at the hotel
- type_of_meal_plan: Type of meal plan booked by the customer:
- Not Selected – No meal plan selected
- Meal Plan 1 – Breakfast
- Meal Plan 2 – Half board (breakfast and one other meal)
- Meal Plan 3 – Full board (breakfast, lunch, and dinner)
- required_car_parking_space: Does the customer require a car parking space? (0 - No, 1- Yes)
- room_type_reserved: Type of room reserved by the customer. The values are ciphered (encoded) by INN Hotels.
- lead_time: Number of days between the date of booking and the arrival date
- arrival_year: Year of arrival date
- arrival_month: Month of arrival date
- arrival_date: Date of the month
- market_segment_type: Market segment designation.
- repeated_guest: Is the customer a repeated guest? (0 - No, 1- Yes)
- no_of_previous_cancellations: Number of previous bookings that were canceled by the customer prior to the current booking
- no_of_previous_bookings_not_canceled: Number of previous bookings not canceled by the customer prior to the current booking
- avg_price_per_room: Average price per day of the reservation; prices of the rooms are dynamic. (in euros)
- no_of_special_requests: Total number of special requests made by the customer (e.g. high floor, view from the room, etc)
- booking_status: Flag indicating if the booking was canceled or not.
Importing necessary libraries and data¶
In [ ]:
import warnings
warnings.filterwarnings("ignore")
# Libraries to help with reading and manipulating data
import pandas as pd
import numpy as np
# libaries to help with data visualization
import matplotlib.pyplot as plt
import seaborn as sns
# Removes the limit for the number of displayed columns
pd.set_option("display.max_columns", None)
# Sets the limit for the number of displayed rows
pd.set_option("display.max_rows", 200)
# setting the precision of floating numbers to 5 decimal points
pd.set_option("display.float_format", lambda x: "%.5f" % x)
# Library to split data
from sklearn.model_selection import train_test_split
# To build model for prediction
import statsmodels.stats.api as sms
from statsmodels.stats.outliers_influence import variance_inflation_factor
import statsmodels.api as sm
from statsmodels.tools.tools import add_constant
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree
# To tune different models
from sklearn.model_selection import GridSearchCV
# To tune different models
from sklearn.model_selection import GridSearchCV
# To get diferent metric scores
from sklearn.metrics import (
f1_score,
accuracy_score,
recall_score,
precision_score,
confusion_matrix,
make_scorer,
roc_auc_score,
)
# To get diferent metric scores
from sklearn.metrics import (
precision_recall_curve,
roc_curve,
)
Data Overview¶
- Observations
- Sanity checks
In [ ]:
#Importing Data Set
hotel = pd.read_csv("/content/drive/MyDrive/Colab Notebooks/INNHotelsGroup.csv")
In [ ]:
# copying data to another variable to avoid any changes to original data
data = hotel.copy()
In [ ]:
#Vieweing The First 5 rows
data.head()
Out[ ]:
In [ ]:
#Viewing The Last 5 rows
data.tail()
Out[ ]:
In [ ]:
#Viewing Shape Of The Data
data.shape
Out[ ]:
The data has 36275 rows and 19 columns.¶
In [ ]:
#Checking the data types of the columns for the dataset
data.info()
The data consist of 1 floating, 13 integer and 5 object data type.¶
In [ ]:
#Checking For Duplicate Values
# checking for duplicate values
data.duplicated().sum()
Out[ ]:
No duplicate values from the data¶
In [ ]:
#Checking For Missing Values
data.isnull().sum()
Out[ ]:
No missing values from the data¶
Dropping The Booking_ID Column First Before Proceeding Forward¶
In [ ]:
data=data.drop("Booking_ID", axis=1)
In [ ]:
data.head()
Out[ ]:
Statistical Summary Of The Data¶
In [ ]:
data.describe().T
Out[ ]:
The average price of a room is around 103 dollars with the highest price around 540 dollars¶
Exploratory Data Analysis (EDA)¶
- EDA is an important part of any project involving data.
- It is important to investigate and understand the data better before building a model with it.
- A few questions have been mentioned below which will help you approach the analysis in the right manner and generate insights from the data.
- A thorough analysis of the data, in addition to the questions mentioned below, should be done.
Leading Questions:
- What are the busiest months in the hotel?
- Which market segment do most of the guests come from?
- Hotel rates are dynamic and change according to demand and customer demographics. What are the differences in room prices in different market segments?
- What percentage of bookings are canceled?
- Repeating guests are the guests who stay in the hotel often and are important to brand equity. What percentage of repeating guests cancel?
- Many guests have special requirements when booking a hotel room. Do these requirements affect booking cancellation?
The below functions need to be defined to carry out the EDA.¶
In [ ]:
def histogram_boxplot(data, feature, figsize=(15, 10), kde=False, bins=None):
"""
Boxplot and histogram combined
data: dataframe
feature: dataframe column
figsize: size of figure (default (15,10))
kde: whether to show the density curve (default False)
bins: number of bins for histogram (default None)
"""
f2, (ax_box2, ax_hist2) = plt.subplots(
nrows=2, # Number of rows of the subplot grid= 2
sharex=True, # x-axis will be shared among all subplots
gridspec_kw={"height_ratios": (0.25, 0.75)},
figsize=figsize,
) # creating the 2 subplots
sns.boxplot(
data=data, x=feature, ax=ax_box2, showmeans=True, color="violet"
) # boxplot will be created and a triangle will indicate the mean value of the column
sns.histplot(
data=data, x=feature, kde=kde, ax=ax_hist2, bins=bins
) if bins else sns.histplot(
data=data, x=feature, kde=kde, ax=ax_hist2
) # For histogram
ax_hist2.axvline(
data[feature].mean(), color="green", linestyle="--"
) # Add mean to the histogram
ax_hist2.axvline(
data[feature].median(), color="black", linestyle="-"
) # Add median to the histogram
In [ ]:
# function to create labeled barplots
def labeled_barplot(data, feature, perc=False, n=None):
"""
Barplot with percentage at the top
data: dataframe
feature: dataframe column
perc: whether to display percentages instead of count (default is False)
n: displays the top n category levels (default is None, i.e., display all levels)
"""
total = len(data[feature]) # length of the column
count = data[feature].nunique()
if n is None:
plt.figure(figsize=(count + 2, 6))
else:
plt.figure(figsize=(n + 2, 6))
plt.xticks(rotation=90, fontsize=15)
ax = sns.countplot(
data=data,
x=feature,
palette="Paired",
order=data[feature].value_counts().index[:n],
)
for p in ax.patches:
if perc == True:
label = "{:.1f}%".format(
100 * p.get_height() / total
) # percentage of each class of the category
else:
label = p.get_height() # count of each level of the category
x = p.get_x() + p.get_width() / 2 # width of the plot
y = p.get_height() # height of the plot
ax.annotate(
label,
(x, y),
ha="center",
va="center",
size=12,
xytext=(0, 5),
textcoords="offset points",
) # annotate the percentage
plt.show() # show the plot
In [ ]:
def stacked_barplot(data, predictor, target):
"""
Print the category counts and plot a stacked bar chart
data: dataframe
predictor: independent variable
target: target variable
"""
count = data[predictor].nunique()
sorter = data[target].value_counts().index[-1]
tab1 = pd.crosstab(data[predictor], data[target], margins=True).sort_values(
by=sorter, ascending=False
)
print(tab1)
print("-" * 120)
tab = pd.crosstab(data[predictor], data[target], normalize="index").sort_values(
by=sorter, ascending=False
)
tab.plot(kind="bar", stacked=True, figsize=(count + 5, 5))
plt.legend(
loc="lower left", frameon=False,
)
plt.legend(loc="upper left", bbox_to_anchor=(1, 1))
plt.show()
In [ ]:
### function to plot distributions wrt target
def distribution_plot_wrt_target(data, predictor, target):
fig, axs = plt.subplots(2, 2, figsize=(12, 10))
target_uniq = data[target].unique()
axs[0, 0].set_title("Distribution of target for target=" + str(target_uniq[0]))
sns.histplot(
data=data[data[target] == target_uniq[0]],
x=predictor,
kde=True,
ax=axs[0, 0],
color="teal",
stat="density",
)
axs[0, 1].set_title("Distribution of target for target=" + str(target_uniq[1]))
sns.histplot(
data=data[data[target] == target_uniq[1]],
x=predictor,
kde=True,
ax=axs[0, 1],
color="orange",
stat="density",
)
axs[1, 0].set_title("Boxplot w.r.t target")
sns.boxplot(data=data, x=target, y=predictor, ax=axs[1, 0], palette="gist_rainbow")
axs[1, 1].set_title("Boxplot (without outliers) w.r.t target")
sns.boxplot(
data=data,
x=target,
y=predictor,
ax=axs[1, 1],
showfliers=False,
palette="gist_rainbow",
)
plt.tight_layout()
plt.show()
Univariate Analysis¶
Observations on lead time¶
In [ ]:
histogram_boxplot(data, "lead_time")
Observations on average price per room¶
In [ ]:
histogram_boxplot(data,"avg_price_per_room")
The average price per room is almost evenly distributed with many outliers¶
In [ ]:
data[data["avg_price_per_room"] == 0]
Out[ ]:
In [ ]:
data.loc[data["avg_price_per_room"] == 0, "market_segment_type"].value_counts()
Out[ ]:
In [ ]:
# Calculating the 25th quantile
Q1 = data["avg_price_per_room"].quantile(0.25)
# Calculating the 75th quantile
Q3 = data["avg_price_per_room"].quantile(0.75)
# Calculating IQR
IQR = Q3 - Q1
# Calculating value of upper whisker
Upper_Whisker = Q3 + 1.5 * IQR
Upper_Whisker
Out[ ]:
In [ ]:
# assigning the outliers the value of upper whisker
data.loc[data["avg_price_per_room"] >= 500, "avg_price_per_room"] = Upper_Whisker
Observations on number of previous booking cancellations¶
In [ ]:
histogram_boxplot(data,"no_of_previous_cancellations")
Observations on number of previous booking not canceled¶
In [ ]:
histogram_boxplot(data,"no_of_previous_bookings_not_canceled")
Observations on number of adults¶
In [ ]:
labeled_barplot(data, "no_of_adults", perc=True)
About 72% of the number of adults that reserve the hotel rooms are 2, 21.2% for only 1 adult and about 6.4% are 3 adults¶
Observations on number of children¶
In [ ]:
labeled_barplot(data,"no_of_children", perc=True)
About 1618 times do memebrs that reserve hotel rooms include 1 child, about 1058 times included 2 children.¶
In [ ]:
# replacing 9, and 10 children with 3
data["no_of_children"] = data["no_of_children"].replace([9, 10], 3)
In [ ]:
data.head()
Out[ ]:
Observations on number of week nights¶
In [ ]:
labeled_barplot(data,"no_of_week_nights", perc=True)
Majority of the customers reserve 1,2,and 3 week nights with 2 being the most reserved number of nights followed by 1 and then 3.¶
Observations on number of weekend nights¶
In [ ]:
labeled_barplot(data,"no_of_weekend_nights", perc=True)
Most of the the customers prefer to reserve only 1 or 2 weekend nights.¶
Observations on required car parking space¶
In [ ]:
labeled_barplot(data,"required_car_parking_space", perc=True)
About 96.9% of the customers require a car parking space¶
Observations on type of meal plan¶
In [ ]:
labeled_barplot(data,"type_of_meal_plan")
Most of the customers(27835) prefer meal plan 1, about 3305 opt in for meal plan 2 and very few opting for meal plan 3. About 5130 does not opt in for any meal plan.¶
Observations on room type reserved¶
In [ ]:
labeled_barplot(data,"room_type_reserved", perc=True)
77.5% of the customers prefer to reserve room type 1.¶
Observations on arrival month¶
In [ ]:
labeled_barplot(data,"arrival_month", perc=True)
The busiests months are October(14.7%), followed by September(12.7%) and Augus(10.5%)t. January has the lowest reservation month(2.8%).¶
Observations on market segment type¶
In [ ]:
labeled_barplot(data,"market_segment_type", perc=True)
The market segment designation is highest for online(64%) and lowest in aviation(0.3%).¶
Observations on number of special requests¶
In [ ]:
labeled_barplot(data,"no_of_special_requests", perc=True)
31.4% of the customers have requested for a special request only once, and very few have request for the 5th time.¶
Observations on booking status¶
In [ ]:
labeled_barplot(data,"booking_status", perc=True)
About 67.2% of the customers did not cancel their booking status, 32.8% cancelled theirs.¶
Encoding Canceled bookings to 1 and Not_Canceled as 0 for further analysis¶
In [ ]:
data["booking_status"] = data["booking_status"].apply(
lambda x: 1 if x == "Canceled" else 0
)
Bivariate Analysis¶
In [ ]:
cols_list = data.select_dtypes(include=np.number).columns.tolist()
plt.figure(figsize=(12, 7))
sns.heatmap(
data[cols_list].corr(), annot=True, vmin=-1, vmax=1, fmt=".2f", cmap="Spectral"
)
plt.show()
Hotel rates are dynamic and change according to demand and customer demographics. Let's see how prices vary across different market segments¶
In [ ]:
plt.figure(figsize=(10, 6))
sns.boxplot(data=data, x="market_segment_type", y="avg_price_per_room", palette="gist_rainbow")
plt.show()
The room prices are averagely higher for online maketing segement, followed by aviation, and offline marketing segment. The least room prices are from complementary market segments.¶
Let's see how booking status varies across different market segments. Also, how average price per room impacts booking status¶
In [ ]:
stacked_barplot(data, "market_segment_type", "booking_status")
Many guests have special requirements when booking a hotel room. Impacts on cancellations¶
In [ ]:
stacked_barplot(data, "no_of_special_requests","booking_status")
Impacts of special requests made by the customers to the prices of a room¶
In [ ]:
plt.figure(figsize=(10, 5))
sns.boxplot(data, x="no_of_special_requests", y="avg_price_per_room", palette="gist_rainbow", showfliers=False)
plt.show()
On average, the higher the speical request the higher the average price except for the customers with 5 special request.¶
Correlation between booking status and average price per room¶
In [ ]:
distribution_plot_wrt_target(data, "avg_price_per_room", "booking_status")
Correlation between booking status and lead time¶
In [ ]:
distribution_plot_wrt_target(data,"booking_status","lead_time")
Dataframe for customers who travel with their families¶
In [ ]:
family_data = data[(data["no_of_children"] >= 0) & (data["no_of_adults"] > 1)]
family_data.shape
Out[ ]:
There are 28441 of the customers who travel with their families¶
In [ ]:
family_data
Out[ ]:
In [ ]:
family_data["no_of_family_members"] = (
family_data["no_of_adults"] + family_data["no_of_children"]
)
In [ ]:
stacked_barplot(family_data,"no_of_family_members","booking_status")
Customers that stay for at least a day at the hotel¶
In [ ]:
stay_data = data[(data["no_of_week_nights"] > 0) & (data["no_of_weekend_nights"] > 0)]
stay_data.shape
Out[ ]:
In [ ]:
stay_data["total_days"] = (
stay_data["no_of_week_nights"] + stay_data["no_of_weekend_nights"]
)
In [ ]:
stay_data
Out[ ]:
In [ ]:
stacked_barplot(stay_data,"total_days", "booking_status")
Busiest months in the hotel.¶
In [ ]:
# grouping the data on arrival months and extracting the count of bookings
monthly_data = data.groupby(["arrival_month"])["booking_status"].count()
# creating a dataframe with months and count of customers in each month
monthly_data = pd.DataFrame(
{"Month": list(monthly_data.index), "Guests": list(monthly_data.values)}
)
# plotting the trend over different months
plt.figure(figsize=(10, 5))
sns.lineplot(data=monthly_data, x="Month", y="Guests")
plt.show()
In [ ]:
monthly_data
Out[ ]:
Percentage of bookings canceled in each month¶
In [ ]:
stacked_barplot(data,"arrival_month","booking_status")
Percentage, July has more reservations being cancelled followed by June and August. The month with least booking cancellation is january due to fewer hotel reservation.¶
Price variation across different months¶
In [ ]:
plt.figure(figsize=(10, 5))
sns.lineplot(data,x="arrival_month",y="avg_price_per_room")
plt.show()
There is a sharp increase in price from the period between February up to May and it remains almost stable till September. It then experiences a sharp decrease from September to December.¶
Data Preprocessing¶
- Missing value treatment (if needed)
- Feature engineering (if needed)
- Outlier detection and treatment (if needed)
- Preparing data for modeling
- Any other preprocessing steps (if needed)
Outlier Detection¶
In [ ]:
#outlier detection using boxplot
numeric_columns = data.select_dtypes(include=np.number).columns.tolist()
# dropping booking_status
numeric_columns.remove("booking_status")
plt.figure(figsize=(15, 12))
for i, variable in enumerate(numeric_columns):
plt.subplot(4, 4, i + 1)
plt.boxplot(data[variable], whis=1.5)
plt.tight_layout()
plt.title(variable)
plt.show()
- There are quite a few outliers in the data.
- However, we will not treat them as they are proper values
In [ ]:
# defining a function to compute different metrics to check performance of a classification model built using statsmodels
def model_performance_classification_statsmodels(
model, predictors, target, threshold=0.5
):
"""
Function to compute different metrics to check classification model performance
model: classifier
predictors: independent variables
target: dependent variable
threshold: threshold for classifying the observation as class 1
"""
# checking which probabilities are greater than threshold
pred_temp = model.predict(predictors) > threshold
# rounding off the above values to get classes
pred = np.round(pred_temp)
acc = accuracy_score(target, pred) # to compute Accuracy
recall = recall_score(target, pred) # to compute Recall
precision = precision_score(target, pred) # to compute Precision
f1 = f1_score(target, pred) # to compute F1-score
# creating a dataframe of metrics
df_perf = pd.DataFrame(
{"Accuracy": acc, "Recall": recall, "Precision": precision, "F1": f1,},
index=[0],
)
return df_perf
In [ ]:
# defining a function to plot the confusion_matrix of a classification model
def confusion_matrix_statsmodels(model, predictors, target, threshold=0.5):
"""
To plot the confusion_matrix with percentages
model: classifier
predictors: independent variables
target: dependent variable
threshold: threshold for classifying the observation as class 1
"""
y_pred = model.predict(predictors) > threshold
cm = confusion_matrix(target, y_pred)
labels = np.asarray(
[
["{0:0.0f}".format(item) + "\n{0:.2%}".format(item / cm.flatten().sum())]
for item in cm.flatten()
]
).reshape(2, 2)
plt.figure(figsize=(6, 4))
sns.heatmap(cm, annot=labels, fmt="")
plt.ylabel("True label")
plt.xlabel("Predicted label")
Preparing data for modeling¶
In [ ]:
X = data.drop(["booking_status"], axis=1)
Y = data["booking_status"]
# adding constant
X = sm.add_constant(X)
X = pd.get_dummies(X, drop_first=True)
# Splitting data in train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.30, random_state=1)
In [ ]:
print("Shape of Training set : ", X_train.shape)
print("Shape of test set : ", X_test.shape)
print("Percentage of classes in training set:")
print(y_train.value_counts(normalize=True))
print("Percentage of classes in test set:")
print(y_test.value_counts(normalize=True))
- We had seen that around 67-68% of observations belongs to class 0 (Not Cancelled) and 32-33% observations belongs to class 1 (Cancelled), and this is preserved in the train and test sets
Building Logistic Regression Model¶
In [ ]:
# fitting logistic regression model
logit = sm.Logit(y_train, X_train.astype(float))
lg = logit.fit()
print(lg.summary())
In [ ]:
print("Training performance:")
model_performance_classification_statsmodels(lg, X_train, y_train)
Out[ ]:
EDA¶
- It is a good idea to explore the data once again after manipulating it.
Univariate Analysis¶
Observations on lead time¶
In [ ]:
histogram_boxplot(data, "lead_time")
Observations on average price per room¶
In [ ]:
histogram_boxplot(data,"avg_price_per_room")
The average price per room is almost evenly distributed with many outliers¶
Observations on number of previous booking cancellations¶
In [ ]:
histogram_boxplot(data,"no_of_previous_cancellations")
Observations on number of previous booking not canceled¶
In [ ]:
histogram_boxplot(data,"no_of_previous_bookings_not_canceled")
Observations on number of adults¶
In [ ]:
labeled_barplot(data, "no_of_adults", perc=True)
About 72% of the number of adults that reserve the hotel rooms are 2, 21.2% for only 1 adult and about 6.4% are 3 adults¶
Observations on number of children¶
In [ ]:
labeled_barplot(data,"no_of_children", perc=True)
Observations on number of week nights¶
In [ ]:
labeled_barplot(data,"no_of_week_nights", perc=True)
Observations on number of weekend nights¶
In [ ]:
labeled_barplot(data,"no_of_weekend_nights", perc=True)
Observations on required car parking space¶
In [ ]:
labeled_barplot(data,"required_car_parking_space", perc=True)
Observations on type of meal plan¶
In [ ]:
labeled_barplot(data,"type_of_meal_plan", perc=True)
Observations on room type reserved¶
In [ ]:
labeled_barplot(data,"room_type_reserved", perc=True)
Observations on arrival month¶
In [ ]:
labeled_barplot(data,"arrival_month", perc=True)
Observations on market segment type¶
In [ ]:
labeled_barplot(data,"market_segment_type", perc=True)
Observations on number of special requests¶
In [ ]:
labeled_barplot(data,"no_of_special_requests", perc=True)
Observations on booking status¶
In [ ]:
labeled_barplot(data,"booking_status", perc=True)
Bivariate Analysis¶
In [ ]:
cols_list = data.select_dtypes(include=np.number).columns.tolist()
plt.figure(figsize=(12, 7))
sns.heatmap(
data[cols_list].corr(), annot=True, vmin=-1, vmax=1, fmt=".2f", cmap="Spectral"
)
plt.show()
Correlation Between Market segment and average price per room¶
In [ ]:
plt.figure(figsize=(10, 6))
sns.boxplot(data=data, x="market_segment_type", y="avg_price_per_room", palette="gist_rainbow")
plt.show()
Correlation between market segment type and booking status¶
In [ ]:
stacked_barplot(data, "market_segment_type", "booking_status")
Correlation between number of special requests and booking status¶
In [ ]:
stacked_barplot(data, "no_of_special_requests","booking_status")
Correlation between number of special requests and average price per room¶
In [ ]:
plt.figure(figsize=(10, 5))
sns.boxplot(data, x="no_of_special_requests", y="avg_price_per_room", palette="gist_rainbow", showfliers=False)
plt.show()
Correlation between average price per room and booking status¶
In [ ]:
distribution_plot_wrt_target(data, "avg_price_per_room", "booking_status")
Correlation between booking status and lead time¶
In [ ]:
distribution_plot_wrt_target(data,"booking_status","lead_time")
Checking Multicollinearity¶
- In order to make statistical inferences from a logistic regression model, it is important to ensure that there is no multicollinearity present in the data.
In [ ]:
#Defining a function to check VIF
def checking_vif(predictors):
vif = pd.DataFrame()
vif["feature"] = predictors.columns
# calculating VIF for each feature
vif["VIF"] = [
variance_inflation_factor(predictors.values, i)
for i in range(len(predictors.columns))
]
return vif
In [ ]:
checking_vif(X_train)
Out[ ]:
In [ ]:
# initial list of columns
cols = X_train.columns.tolist()
# setting an initial max p-value
max_p_value = 1
while len(cols) > 0:
# defining the train set
x_train_aux = X_train[cols]
# fitting the model
model = sm.Logit(y_train, x_train_aux).fit(disp=False)
# getting the p-values and the maximum p-value
p_values = model.pvalues
max_p_value = max(p_values)
# name of the variable with maximum p-value
feature_with_p_max = p_values.idxmax()
if max_p_value > 0.05:
cols.remove(feature_with_p_max)
else:
break
selected_features = cols
print(selected_features)
In [ ]:
X_train1 = X_train[selected_features]
X_test1 = X_test[selected_features]
In [ ]:
logit1 = sm.Logit(y_train,X_train1.astype(float))
lg1 = logit1.fit()
print(lg1.summary())
- No categorical feature has p-value greater than 0.05.
Interpretation¶
- The coefficient of number of adults, number of children, number of weekend nights, number of week nights, lead time, number of previous cancellations, average price, are all positive. Increase in that will imply no cancellation of reservation.
- The coeeficient of market segment type offline, market segment type corporate, room type reserved, number of special request, required parking space are all negative therefore increasing them will imply cancellation of reservation.
Model performance evaluation¶
In [ ]:
print("Training performance:")
model_performance_classification_statsmodels(lg1,X_train1, y_train)
Out[ ]:
- No significant change in the model performance.
In [ ]:
# converting coefficients to odds
odds = np.exp(lg1.params)
# finding the percentage change
perc_change_odds = (np.exp(lg1.params) - 1) * 100
# removing limit from number of columns to display
pd.set_option("display.max_columns", None)
# adding the odds to a dataframe
pd.DataFrame({"Odds": odds, "Change_odd%": perc_change_odds}, index=X_train1.columns).T
Out[ ]:
Checking model performance on the training set¶
In [ ]:
# creating confusion matrix
confusion_matrix_statsmodels(lg1, X_train1, y_train)
In [ ]:
print("Training performance:")
log_reg_model_train_perf = model_performance_classification_statsmodels(lg1,X_train1, y_train) ## Complete the code to check performance on X_train1 and y_train
log_reg_model_train_perf
Out[ ]:
- The recall scores in the training data is around 0.63.
- The accuracy on the training data is around 0.81.
ROC-AUC¶
- ROC-AUC on training set
In [ ]:
logit_roc_auc_train = roc_auc_score(y_train, lg1.predict(X_train1))
fpr, tpr, thresholds = roc_curve(y_train, lg1.predict(X_train1))
plt.figure(figsize=(7, 5))
plt.plot(fpr, tpr, label="Logistic Regression (area = %0.2f)" % logit_roc_auc_train)
plt.plot([0, 1], [0, 1], "r--")
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.01])
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("Receiver operating characteristic")
plt.legend(loc="lower right")
plt.show()
- The model is performing well on the training set.
Model Performance Improvement¶
- Let's see if the recall score can be improved further, by changing the model threshold using AUC-ROC Curve.
In [ ]:
# Optimal threshold as per AUC-ROC curve
# The optimal cut off would be where tpr is high and fpr is low
fpr, tpr, thresholds = roc_curve(y_train, lg1.predict(X_train1))
optimal_idx = np.argmax(tpr - fpr)
optimal_threshold_auc_roc = thresholds[optimal_idx]
print(optimal_threshold_auc_roc)
In [ ]:
# creating confusion matrix
confusion_matrix_statsmodels(
lg1, X_train1, y_train, threshold=optimal_threshold_auc_roc,
)
In [ ]:
# checking model performance for this model
log_reg_model_train_perf_threshold_auc_roc = model_performance_classification_statsmodels(
lg1, X_train1, y_train, threshold=optimal_threshold_auc_roc
)
print("Training performance:")
log_reg_model_train_perf_threshold_auc_roc
Out[ ]:
- Recall and F1 of model has increased but the other metrics have reduced.
- The model is still giving a good performance.
Let's use Precision-Recall curve and see if we can find a better threshold¶
In [ ]:
y_scores = lg1.predict(X_train1)
prec, rec, tre = precision_recall_curve(y_train, y_scores,)
def plot_prec_recall_vs_tresh(precisions, recalls, thresholds):
plt.plot(thresholds, precisions[:-1], "b--", label="precision")
plt.plot(thresholds, recalls[:-1], "g--", label="recall")
plt.xlabel("Threshold")
plt.legend(loc="upper left")
plt.ylim([0, 1])
plt.figure(figsize=(10, 7))
plot_prec_recall_vs_tresh(prec, rec, tre)
plt.show()
In [ ]:
# setting the threshold
optimal_threshold_curve = 0.42
Checking model performance on training set¶
In [ ]:
# creating confusion matrix
confusion_matrix_statsmodels(
lg1,X_train1, y_train, threshold=optimal_threshold_curve,
)
In [ ]:
log_reg_model_train_perf_threshold_curve = model_performance_classification_statsmodels(
lg1, X_train1, y_train, threshold=optimal_threshold_curve
)
print("Training performance:")
log_reg_model_train_perf_threshold_curve
Out[ ]:
- The accuracy of the model has increased, whereas precision and recall are all around 0.7.
- The overall performance of the model is good.
Let's check the performance on the test set¶
- Using model with default threshold
In [ ]:
# creating confusion matrix
confusion_matrix_statsmodels( lg1,X_test1, y_test, threshold=optimal_threshold_curve,)
In [ ]:
log_reg_model_test_perf = model_performance_classification_statsmodels(lg1, X_test1, y_test, threshold=optimal_threshold_curve)
print("Test performance:")
log_reg_model_test_perf
Out[ ]:
- The recall scores in the test data is around 0.70.
- The accuracy on the test data is around 0.80.
- The precision on the test data is around 0.69.
- ROC curve on test set
In [ ]:
logit_roc_auc_train = roc_auc_score(y_test, lg1.predict(X_test1))
fpr, tpr, thresholds = roc_curve(y_test, lg1.predict(X_test1))
plt.figure(figsize=(7, 5))
plt.plot(fpr, tpr, label="Logistic Regression (area = %0.2f)" % logit_roc_auc_train)
plt.plot([0, 1], [0, 1], "r--")
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.01])
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("Receiver operating characteristic")
plt.legend(loc="lower right")
plt.show()
- The model is performing well on the test set.
Using model with threshold=0.37¶
In [ ]:
# creating confusion matrix
confusion_matrix_statsmodels(lg1, X_test1, y_test, threshold=optimal_threshold_auc_roc,)
In [ ]:
# checking model performance for this model
log_reg_model_test_perf_threshold_auc_roc = model_performance_classification_statsmodels(
lg1, X_test1, y_test, threshold=optimal_threshold_auc_roc
)
print("Test performance:")
log_reg_model_test_perf_threshold_auc_roc
Out[ ]:
- The score on the accuracy data reduced whilest the score of recall increased
- F1 score also increased
- The model is performing well generally
Using model with threshold = 0.42¶
In [ ]:
# creating confusion matrix
confusion_matrix_statsmodels(lg1, X_test1, y_test, threshold=optimal_threshold_auc_roc)
In [ ]:
log_reg_model_test_perf_threshold_curve = model_performance_classification_statsmodels(
lg1, X_test1, y_test, threshold=optimal_threshold_curve
)
print("Test performance:")
log_reg_model_test_perf_threshold_curve
Out[ ]:
Model performance summary¶
In [ ]:
# training performance comparison
models_train_comp_df = pd.concat(
[
log_reg_model_train_perf.T,
log_reg_model_train_perf_threshold_auc_roc.T,
log_reg_model_train_perf_threshold_curve.T,
],
axis=1,
)
models_train_comp_df.columns = [
"Logistic Regression-default Threshold",
"Logistic Regression-0.37 Threshold",
"Logistic Regression-0.42 Threshold",
]
print("Training performance comparison:")
models_train_comp_df
Out[ ]:
In [ ]:
# test performance comparison
models_test_comp_df = pd.concat(
[
log_reg_model_test_perf.T,
log_reg_model_test_perf_threshold_auc_roc.T,
log_reg_model_test_perf_threshold_curve.T,
],
axis=1,
)
models_test_comp_df.columns = [
"Logistic Regression-default Threshold",
"Logistic Regression-0.37 Threshold",
"Logistic Regression-0.42 Threshold",
]
print("Test set performance comparison:")
models_test_comp_df
Out[ ]:
- Almost all the three models are performing well on both training and test data without the problem of overfitting
- The model with a default threshold (0.37) is giving the best F1 score. Therefore it can be selected as the final model
Decision Tree
- Data Preparation for modeling (Decision Tree)
In [ ]:
X = data.drop(["booking_status"], axis=1)
Y = data["booking_status"]
X = pd.get_dummies(X, drop_first=True)
# Splitting data in train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.30, random_state=1)
In [ ]:
print("Shape of Training set : ", X_train.shape)
print("Shape of test set : ", X_test.shape)
print("Percentage of classes in training set:")
print(y_train.value_counts(normalize=True))
print("Percentage of classes in test set:")
print(y_test.value_counts(normalize=True))
- We had seen that around 67% of observations belongs to class 0 (Not Cancelled) and 33% observations belongs to class 1 (Cancelled), and this is preserved in the train and test sets
In [ ]:
# defining a function to compute different metrics to check performance of a classification model built using sklearn
def model_performance_classification_sklearn(model, predictors, target):
"""
Function to compute different metrics to check classification model performance
model: classifier
predictors: independent variables
target: dependent variable
"""
# predicting using the independent variables
pred = model.predict(predictors)
acc = accuracy_score(target, pred) # to compute Accuracy
recall = recall_score(target, pred) # to compute Recall
precision = precision_score(target, pred) # to compute Precision
f1 = f1_score(target, pred) # to compute F1-score
# creating a dataframe of metrics
df_perf = pd.DataFrame(
{"Accuracy": acc, "Recall": recall, "Precision": precision, "F1": f1,},
index=[0],
)
return df_perf
In [ ]:
def confusion_matrix_sklearn(model, predictors, target):
"""
To plot the confusion_matrix with percentages
model: classifier
predictors: independent variables
target: dependent variable
"""
y_pred = model.predict(predictors)
cm = confusion_matrix(target, y_pred)
labels = np.asarray(
[
["{0:0.0f}".format(item) + "\n{0:.2%}".format(item / cm.flatten().sum())]
for item in cm.flatten()
]
).reshape(2, 2)
plt.figure(figsize=(6, 4))
sns.heatmap(cm, annot=labels, fmt="")
plt.ylabel("True label")
plt.xlabel("Predicted label")
Building Decision Tree Model¶
In [ ]:
model = DecisionTreeClassifier(random_state=1)
model.fit(X_train, y_train)
Out[ ]:
Checking model performance on training set¶
In [ ]:
confusion_matrix_sklearn(model, X_train, y_train)
In [ ]:
decision_tree_perf_train = model_performance_classification_sklearn(
model, X_train, y_train
)
decision_tree_perf_train
Out[ ]:
Checking model performance on test set¶
In [ ]:
confusion_matrix_sklearn(model, X_test, y_test)
In [ ]:
decision_tree_perf_test = model_performance_classification_sklearn(model, X_test, y_test) ## Complete the code to check performance on test set
decision_tree_perf_test
Out[ ]:
Checking the important features before pruning the tree¶
In [ ]:
feature_names = list(X_train.columns)
importances = model.feature_importances_
indices = np.argsort(importances)
plt.figure(figsize=(8, 8))
plt.title("Feature Importances")
plt.barh(range(len(indices)), importances[indices], color="violet", align="center")
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel("Relative Importance")
plt.show()
- We can see the Lead time and average price per room plays a significant row in the decision tree
Pruning the tree¶
- Pre-Pruning
In [ ]:
# Choosing the type of classifier.
estimator = DecisionTreeClassifier(random_state=1, class_weight="balanced")
# Grid of parameters to choose from
parameters = {
"max_depth": np.arange(2, 7, 2),
"max_leaf_nodes": [50, 75, 150, 250],
"min_samples_split": [10, 30, 50, 70],
}
# Type of scoring used to compare parameter combinations
acc_scorer = make_scorer(f1_score)
# Run the grid search
grid_obj = GridSearchCV(estimator, parameters, scoring=acc_scorer, cv=5)
grid_obj = grid_obj.fit(X_train, y_train)
# Set the clf to the best combination of parameters
estimator = grid_obj.best_estimator_
# Fit the best algorithm to the data.
estimator.fit(X_train, y_train)
Out[ ]:
Checking performance on training set¶
In [ ]:
confusion_matrix_sklearn(model, X_train, y_train)
In [ ]:
decision_tree_tune_perf_train = model_performance_classification_sklearn(model, X_train, y_train) ## Complete the code to check performance on train set
decision_tree_tune_perf_train
Out[ ]:
- The model has been able to perfectly classify all the data points on the training set.
- There are 0 errors on the training set, each sample has been classified correctly.
- Decision tree will continue to grow and classify each data point correctly if no restrictions are applied as the trees will learn all the patterns in the training set.
- This generally leads to overfitting of the model as Decision Tree will perform well on the training set but will fail to replicate the performance on the test set.
Checking performance on test set¶
In [ ]:
confusion_matrix_sklearn(model, X_test, y_test)
In [ ]:
decision_tree_tune_perf_test = model_performance_classification_sklearn(model, X_test, y_test)
decision_tree_tune_perf_test
Out[ ]:
- There is a huge disparity in performance of model on training set and test set, which suggests that the model is overfiiting.
Visualizing the Decision Tree¶
In [ ]:
feature_names = list(X_train.columns)
importances = estimator.feature_importances_
indices = np.argsort(importances)
Pre-Pruned Tree¶
In [ ]:
plt.figure(figsize=(20, 10))
out = tree.plot_tree(
estimator,
feature_names=feature_names,
filled=True,
fontsize=9,
node_ids=False,
class_names=None,
)
# below code will add arrows to the decision tree split if they are missing
for o in out:
arrow = o.arrow_patch
if arrow is not None:
arrow.set_edgecolor("black")
arrow.set_linewidth(1)
plt.show()
In [ ]:
# Text report showing the rules of a decision tree -
print(tree.export_text(estimator, feature_names=feature_names, show_weights=True))
- If the lead time is less than or equal to 151.50, number of special request is less than or equal to 0.50, market segment type online is less than or equal to 0.50, then the hotel reservation is likely to be cancelled.
In [ ]:
# importance of features in the tree building
importances = estimator.feature_importances_
indices = np.argsort(importances)
plt.figure(figsize=(8, 8))
plt.title("Feature Importances")
plt.barh(range(len(indices)), importances[indices], color="violet", align="center")
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel("Relative Importance")
plt.show()
- Over here lead time, market segment type online and number of special request are the important features.
Decision Tree (Post pruning)¶
- Cost Complexity Pruning
In [ ]:
clf = DecisionTreeClassifier(random_state=1, class_weight="balanced")
path = clf.cost_complexity_pruning_path(X_train, y_train)
ccp_alphas, impurities = abs(path.ccp_alphas), path.impurities
In [ ]:
pd.DataFrame(path)
Out[ ]:
In [ ]:
fig, ax = plt.subplots(figsize=(10, 5))
ax.plot(ccp_alphas[:-1], impurities[:-1], marker="o", drawstyle="steps-post")
ax.set_xlabel("effective alpha")
ax.set_ylabel("total impurity of leaves")
ax.set_title("Total Impurity vs effective alpha for training set")
plt.show()
Training a decision tree using effective alphas¶
In [ ]:
clfs = []
for ccp_alpha in ccp_alphas:
clf = DecisionTreeClassifier(
random_state=1, ccp_alpha=ccp_alpha, class_weight="balanced"
)
clf.fit(X_train, y_train)
clfs.append(clf)
print(
"Number of nodes in the last tree is: {} with ccp_alpha: {}".format(
clfs[-1].tree_.node_count, ccp_alphas[-1]
)
)
In [143]:
clfs = clfs[:-1]
ccp_alphas = ccp_alphas[:-1]
node_counts = [clf.tree_.node_count for clf in clfs]
depth = [clf.tree_.max_depth for clf in clfs]
fig, ax = plt.subplots(2, 1, figsize=(10, 7))
ax[0].plot(ccp_alphas, node_counts, marker="o", drawstyle="steps-post")
ax[0].set_xlabel("alpha")
ax[0].set_ylabel("number of nodes")
ax[0].set_title("Number of nodes vs alpha")
ax[1].plot(ccp_alphas, depth, marker="o", drawstyle="steps-post")
ax[1].set_xlabel("alpha")
ax[1].set_ylabel("depth of tree")
ax[1].set_title("Depth vs alpha")
fig.tight_layout()
- The number of nodes and tree depth decreases as alpha increases.
F1 Score vs alpha for training and testing sets¶
In [144]:
f1_train = []
for clf in clfs:
pred_train = clf.predict(X_train)
values_train = f1_score(y_train, pred_train)
f1_train.append(values_train)
f1_test = []
for clf in clfs:
pred_test = clf.predict(X_test)
values_test = f1_score(y_test, pred_test)
f1_test.append(values_test)
In [145]:
fig, ax = plt.subplots(figsize=(15, 5))
ax.set_xlabel("alpha")
ax.set_ylabel("F1 Score")
ax.set_title("F1 Score vs alpha for training and testing sets")
ax.plot(ccp_alphas, f1_train, marker="o", label="train", drawstyle="steps-post")
ax.plot(ccp_alphas, f1_test, marker="o", label="test", drawstyle="steps-post")
ax.legend()
plt.show()
In [146]:
index_best_model = np.argmax(f1_test)
best_model = clfs[index_best_model]
print(best_model)
Checking performance on training set¶
In [147]:
confusion_matrix_sklearn(best_model, X_train, y_train)
In [148]:
decision_tree_post_perf_train = model_performance_classification_sklearn(
best_model, X_train, y_train
)
decision_tree_post_perf_train
Out[148]:
Checking performance on test set¶
In [149]:
confusion_matrix_sklearn(best_model, X_test, y_test)
In [153]:
decision_tree_post_test = model_performance_classification_sklearn(
best_model, X_test, y_test
)
decision_tree_post_test
Out[153]:
In [154]:
plt.figure(figsize=(20, 10))
out = tree.plot_tree(
best_model,
feature_names=feature_names,
filled=True,
fontsize=9,
node_ids=False,
class_names=None,
)
for o in out:
arrow = o.arrow_patch
if arrow is not None:
arrow.set_edgecolor("black")
arrow.set_linewidth(1)
plt.show()
In [155]:
# Text report showing the rules of a decision tree -
print(tree.export_text(best_model, feature_names=feature_names, show_weights=True))
- Similar observation from the pre-pruned tree is also matching with the decision tree rules of the post pruned tree
In [156]:
importances = best_model.feature_importances_
indices = np.argsort(importances)
plt.figure(figsize=(12, 12))
plt.title("Feature Importances")
plt.barh(range(len(indices)), importances[indices], color="violet", align="center")
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel("Relative Importance")
plt.show()
- Lead time, market segment online, average price per room, and number of special request are the most important features seen.
Comparison of Models and Final Model Selection¶
In [157]:
# training performance comparison
models_train_comp_df = pd.concat(
[
decision_tree_perf_train.T,
decision_tree_tune_perf_train.T,
decision_tree_post_perf_train.T,
],
axis=1,
)
models_train_comp_df.columns = [
"Decision Tree sklearn",
"Decision Tree (Pre-Pruning)",
"Decision Tree (Post-Pruning)",
]
print("Training performance comparison:")
models_train_comp_df
Out[157]:
In [161]:
# test performance comparison
models_test_comp_df = pd.concat(
[
decision_tree_perf_test.T,
decision_tree_tune_perf_test.T,
decision_tree_post_test.T,
],
axis=1,
)
models_train_comp_df.columns = [
"Decision Tree sklearn",
"Decision Tree (Pre-Pruning)",
"Decision Tree (Post-Pruning)",
]
print("Test performance comparison:")
models_test_comp_df
Out[161]:
- Decision tree models with pre-pruning and post-pruning both are giving equally high recall scores on both training and test sets.
- However, we will choose the post-pruned tree as the best model since it is giving a slightly high recall score on the train and test sets than the pre-pruned tree.
Actionable Insights and Recommendations¶
- What profitable policies for cancellations and refunds can the hotel adopt?
- What other recommedations would you suggest to the hotel?
- Lead time, market segment online, average price per room, and number of special request are the most important features predicting whether a hptel reservation will get cancelled or not
- The model built can be used to predict if an hotel reservation is going to get cancelled or not and can correctly identify 85.6% of the reservation cancellation
- From the decision tree, weve learned that ff the lead time is less than or equal to 151.50, number of special request is less than or equal to 0.50, market segment type online is less than or equal to 0.50, then the hotel reservation is likely to be cancelled.
- The company should give a vigilant eye for these values in order to detect hotel reservation cancellation.
The hotel should give a %50 penalty for cancellation of reservation that is due on the exact day a customer is suppose to report.
The company should use more data for the analysis to get more reliable results
- As the variable used vary with the type of operation being performed, the company can look to build separate models for each different type of operation